Performance evaluation of Web Information Retrieval Systems and its application to e-Business

The evaluation of traditional IR systems is performed in an ideal situation, without workload in the IR system, however the Web IR systems are under different workload levels through the time. Therefore, the performance evaluation must be done considering several workload environments. For this purpose, we have designed and developed USim, a tool for the performance evaluation of Web IR systems based on the simulation of users’ behaviour. This work is based on several previous theoretical analyses of the users’ behaviour while using Web search systems. This simulation tool helps in the performance evaluation of the Web IR systems in two different ways: comparing the response times for different search algorithms or engines (calculating the response times for each search engine) and estimating the saturation threshold of the system. The latter point is especially important for guaranteeing an appropriate quality of service to all the users, and therefore, its applications to the e-Business of the IR system are quite significant during the whole life of the system. NOTE: This work has been partially supported by the CICYT (TIC2001-0547).